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ABSTRACT: 

To develop and sustain quality in higher education, the integrated role of course evaluation survey (CES) among 
students in academic programs is fully understood. An original CES questionnaire (CESQ) involving 24 closed 
questions (items), prescribed by the National Commission for Academic Accreditation & Assessment (NCAAA), is 
being used presently at all colleges of our university. This article aims to deal with optimized reduction of items in this 
CESQ. To guide this through scientific research, methodological results under exploratory analysis of existing CES 
data available for College of Nursing are used. The observed collinearity among the items of the CESQ also justifies 
the demand of optimizing it by the university community. A critical comparative appraisal of the analytical results of 
the existing data sets (considering exploratory sub sets of items) provided an optimum set of 15 items that reflects 
relative merits in terms of required attributes: viz., reduced problem of collinearity among items; retained relationship 
among them; appropriateness regarding extracted factors as well as their independence; negligible change in overall 
reliability of questionnaire; and also in proportion of total variance being explained by the underlying factors. This 15 
items CESQ also holds similar merits in comparison to the 24 items CESQ. 

Keywords: academic program, course evaluation survey, questionnaire, item reduction, optimization, collinearity, 


exploratory analysis, reliability 


BACKGROUND 

To develop and maintain high quality in higher 
education, various types of evaluation surveys among 
students play an important role. Since the year 2009, 
Course Evaluation Surveys (CES) have been 
conducted in various undergraduate programs of our 
university. They are Colleges of Nursing, Medicine, 
Dentistry and Engineering. By the Deanship of Quality 
and Academic Accreditation, a CES schedule was 
prepared and executed in consultation with the 
concemed college authority (i.e. deans and course 
coordinators). As a priority, data were analyzed and 
reported accurately and appropriately. Furthermore, 
validation of the results required detailed discussion 
with college authorities. The findings were then widely 
disseminated among corresponding faculty and 
students. As part of their feedback, the colleges 
repeatedly expressed and emphasized two major 
concerns about the existing CES questionnaire 
(English version). First, it should be optimally reduced 
in terms of the number of closed questions (i.e. items). 
Secondly, English not being a mother tongue, further 
clarity of questions to students can enhance the quality, 
accuracy and reliability of data. 


It was this constructive feedback, reflecting real needs 
of the institution, which triggered the conceptualization 
of this research study: viz. optimum reduction of items 
in a CES questionnaire. A range of literature is 


available on development of new _ evaluation 
questionnaires; including CES [1]. In order to 
incorporate real needs of or changes in institutions or 
programs, many authors have also emphasized the 
need to occasionally review existing questionnaires 
that involves possible inclusion as well as exclusion of 
the items. In particular, item reduction in existing 
questionnaire can be mandated by the observed needs 
of institutions [2,3] 


On the one hand, the steps involved in reviewing an 
existing questionnaire are different from those 
involved in developing a new questionnaire. On the 
other hand, the steps involved in optimum reduction of 
items in existing questionnaire are different from those 
involved in its review. Therefore, it is necessary to 
emphasize that this study involves optimum reduction 
of items from an existing questionnaire. It is neither 
the developing of a new questionnaire, nor reviewing 
an existing one. 


In summary, this article deals with an optimum 
reduction of items in an existing original CES 
questionnaire consisting of 24 items (Appendix 1). For 
this, as an attempt to retain the structure and 
psychometric evidence (such as reliability or factor 
analysis) of the original questionnaire in reduced 
questionnaire, results under an exploratory analysis of 
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existing CES data have been used as a supportive 
evidence. 


MATERIALS AND METHODS 

Properly worded items and their appropriate ordering 
may provide more valid information [1,4]. Therefore, 
prior to the process of optimized item reduction, each 
item of the existing (i.e. original) CES questionnaire 
was further clarified by means of changing words or 
sentences or both. Further, all closed items were re- 
arranged, listing them sequentially within five areas. 
This also helped in prioritization regarding selection of 
one of the highly correlated items under this task, 
especially to retain the structure of the questionnaire. 


Keeping in view of the goal under this study, instead of 
relying on only logical and pragmatic approach, 
exploratory analysis of existing CES data was also 
considered. This was driven by the imperative to retain 
the theoretical structure as well as analytical power of 
the original questionnaire. In other words, while 
performing item reduction, care was taken to retain the 
structure and psychometric evidence (such as 
reliability or factor analysis) of the questionnaire. For 
completeness, the existing data used in exploratory 
analysis as well as analytical methods are briefly 
described in the following sections. 


Data 

For exploration, existing CES data from students in 
undergraduate program of the College of Nursing were 
used. Because of limited sample size of available CES 
data under academic programs of other colleges 
(Medicine, Engineering & Dentistry), its consideration 
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remained inappropriate for valid exploration. In 
addition, in the College of Nursing, the CES 
questionnaires were complete in all respects where as 
those in some colleges (e.g. College of Medicine) were 
incomplete in relation to item number 14 “Gave fair 
grade to my Continuous Assessments [tests & 
assignments]. This College had coverage of a total of 
15 courses related to students of semester 1 to 4 during 
the academic year 2008/09 (Table 1). 


Keeping in view of the required appropriateness of 
sample size regarding factor analysis [5,6], the CES 
data related to courses covered at first instance by 
students of each semester were further pooled together. 
Likewise, it was adopted for successive coverage also. 
Thus, this consideration helped in maintaining good to 
very good adequacy of required sample size [6]. This 
adequacy further gets reflected by the observed higher 
magnitude of factor loadings as well as higher 
communalities under factor analysis [5,7] and Kaiser — 
Meyer — Olkin measure (KMO) of sampling adequacy 
[8,9]. In addition, the required assumption related to 
independence of records also remained intact. Finally, 
four independent study groups (Table1) were explored 
from the College of Nursing. 


Table 1: The Courses of B.Sc. Nursing, College of Nursing, used under this exploration 


Course 
Study Group | Semester Heo sue 
Course specific | Group 

1 Applied Medical Physics 112 

1 2 Human Biology 107 410 
3 Anatomy & Physiology I 094 
4 Fundamental of Nursing 097 
1 Chemistry 111 

2 2 Medical Terminology 113 410 
3 Biochemistry 095 
4 Nutrition 091 
1 General English I 112 

3 2 HR & Communication Skills 113 All 
3 Microbiology & Parasitology 095 
4 Pathology 091 
1 Introduction & Bioethics in Nursing | 113 

4 2 IT for Nurse 107 308 
4 Pharmacology 088 
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Table 2: Item Specific Descriptive Statistics 


Study Group 
1 (n=410) 2 (n=410) 3 (n=411) 4 (n=308) 
= gg = g = = 8 a = g re 

tem nA 2 = Hn A =e Hn MIF n FSH MIF HD FE GH M 
At the start of the course, I was made clear about: 

1 |41005 4 4 -11 1.1 [350.07 4 5. -0.5 -1.1|3.90.005 4 4 -0.8 0.1 )3.10.06 3 3. -0.1 -0.7 
2 (390.05 4 4 -0.6 0.0 |3.2 0.063 4 -0.3 -0.9/4 0.05 4 4 -0.6 -0.2/2.80.063 3 O.1 -04 
3 3.70.05 4 4 -0.5 -0.4)3.1 0.07 3 3 -0.1 -1.1]3.60.06 4 4 -0.3 -0.8|2.8 0.06 3 3 0.3 -0.5 
During this course, my instructors: 

4 |3.80.05 4 4 -0.7 0.0 |3.2 0.06 3 4 -0.3 -1.0)3.7 0.05 4 4. -0.6 -0.1 3.1 0.06 3 3. -0.1 -0.6 
5 |3.70.05 4 4. -0.5 -0.3 |3.10.07 3 4 -0.2 -1.2|3.70.05 4 4 -05 -0.6/|2.8 0.063 3 0.0 -0.6 
6 |3.80.05 4 4. -0.7 -0.2/340.07 4 5 -04 -1.1]3.70.06 4 4 -0.6 -0.5]2.8 0.073 3 0.1 -0.9 
7 |3.90.05 4 4 -0.8 0.3 3.50.07 4 4 -0.5 -0.9|3.8 0.05 4 4 -0.6 -0.2|3.1 0.063 3. -0.2 -0.6 
8 |3.70.005 4 4 -0.8 0.1 |2.90.07 3 2 O.1 -1.2)3.60.06 4 4 -04 -0.7]2.8 0.073 3 0.0 -0.8 
9 {3.80.05 4 4 -0.7 0.0 /3.20.07 3 4 -0.3 -1.1|3.8 0.05 4 4 -0.6 -0.3 3.10.06 3 3. -0.2 -0.7 
10 |3.70.05 4 4 -0.5 -0.3|3.3 0.07 3 4 -0.3 -1.1]3.60.06 4 4 -0.5 -0.5]3.0 0.06 3 3 0.0 -0.7 
11 |3.60.05 4 4. -0.6 -0.3 |3.3 0.07 3 5 -0.3 -1.1]3.60.05 4 4 -0.5 -0.4)2.9 0.063 3 0.0 -0.7 
12 |3.70.005 4 4 -05 -0.4/3.2 0.07 3 4 -0.2 -1.2)3.5 0.064 3. -0.3 -0.7/2.8 0.063 3 O.1 -0.6 
13. |3.7 0.05 4 4 -0.6 -0.3 |3.2 0.07 3 5 -0.2 -1.2|3.60.06 4 4 -04 -0.6)2.80.07 3 3 0.1 -0.9 
14 |3.70.005 4 4 -0.5 -0.3 |3.1 0.07 3 4 -0.2 -1.3|3.5 0.06 4 4 -04 -0.7)2.70.06 3 3 0.2 -0.7 
15 |3.50.005 4 4 -04 -0.4/3.10.07 3 4 -0.1 -1.3|3.5 0.0064 4 -04 -0.8)26 0.063 3 0.2 -0.9 
During this course, my college/department: 

16 |3.70.06 4 4. -0.6 -0.3 |3.2 0.06 3 4 -0.2 -1.1]3.60.06 4 4 -0.5 -0.6/3.00.07 3 3 0.0 -0.8 
17 |3.80.05 4 4. -0.7 -0.2|3.2 0.07 3 4 -0.2 -1.2)3.60.06 4 4 -04 -0.6]3.1 0.07 3 3 -0.1 -0.7 
18 |3.70.05 4 4 -0.6 -0.1/3.2 0.07 3 4 -0.3 -1.1]3 0.0063 4 -0.3 -0.9)2.8 0.063 3 0.0 -0.7 
19 |3.70.05 4 4. -0.6 -0.1/3.30.073 4 -04-1.0/4 0.05 4 4 -0.3 -0.7|2.9 0.06 3 3 0.0 -0.7 
Evaluation of the course: this course helped me to: 

20 |3.8 0.06 4 4 -0.7 -0.2/340.07 4 5 -04-1.3/4 0.064 5. -0.7 -0.3)3.1 0.07 3 3. -0.1 -0.6 
21 |3.70.05 4 4. -0.6 -0.4/3.00.07 3 4. -0.1 -1.2|3.5 0.06 4 4 -04 -0.6)2.9 0.06 3 3. -0.1 -0.7 
22 |3.70.05 4 4 -0.5 -0.4/3.00.07 3 3 0.0 -12/3 0.064 4 -0.3 -0.8)2.90.07 3 3 0.1 -0.7 
23 13.60.05 4 4 -05 -0.4/2.8 0.063 3 O.1 -1.1)3 0.0063 4 -0.3 -0.8)/2.8 0.063 3 O.1 -0.6 
Overall evaluation 

24 |3.70.05 4 4. -0.7 -0.1/3.2 0.07 3 4 = -0.3 -1.3|3.6 0.06 4 4 -0.5 -0.6)2.70.07 3 3 0.2 -0.8 
Analytical methods of mean, median, mode, skewness & kurtosis. The 


For the exploratory analysis, although each item was 
observed on a Likert scale (5 = “strongly agree”; 4 = 
“agree” ; 3 = “true sometimes”; 2 = “disagree”; | = 
“strongly disagree”), it was considered to be measured 
as on interval scale [10]. Further, normality of the 
responses to an item was examined through descriptive 
statistics (Table 2). They include mean, standard error 


presence of normality deviation, as evident through 
observed skewness and kurtosis, may be mainly 
attributed to the considered large sets of student scores. 
It may be pointed out here that these numerical ways 
of determining non- normality are very sensitive to the 
considered numbers of scores [11]. Since correlation, 
factor analysis, and related linear techniques are 
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known to be relatively robust against non-extreme 
deviations from normality [12,13], confinement of 
normality presumption based on only skewness and 
kurtosis may be deceptive in the present case. Hence, 
as a pragmatic approach, an eyeball test of first four 
descriptive statistics of evaluation scores on each item 
was used to establish justification in presuming its 
required normality assumption. None of the items in 
this study had extreme deviations from normality, 
hence was not deleted on this basis. 


To begin with, the correlation matrix dealing with 
simple correlation of each item with all other items 
included in CES questionnaire was examined for each 
study group. To further emphasize, as evident from 
these results, one can also visualize the extent of 
relationship of each item with the respective overall 
satisfaction item. The presence of a positive and high 
to very high correlation, for numerous pairs of items 
(especially second study group), indicates possible 
presence of multicollinearity (Appendix 2). As a 
matter of fact, the items have to be intercorrelated, but 
the presence of too high correlation between items may 
cause difficulties in determining the unique 
contribution of an item to a factor [7]. Hence, instead 
of deleting one of them abruptly, it was decided to 
delete one of the highly correlated items more 
objectively based on relative merits observed under 
exploratory factor analysis. 


The inetercorrelations between the items were 
observed in two steps. First, a significant Bartlett’s test 
of sphericity just reveals that there is intercorrelation 
between the items. Second, the determinant of 
correlation matrix less than 0.00001 further reveals 
that there is multicollinearity [7]. The intercorrelation 
between the items is a requirement. However, to 
overcome the problem of collinearity, items involving 
a correlation of 0.70 and more (Appendix 2) were 
considered collinear [14]. For exploration, at a time, 
only one of the two collinear items in such pairs was 
needed to be considered. Keeping in view of the 
presence of more number of such pairs, a series of 
exploratory subsets of items were generated to carry 
out further exploratory analysis. 


Under second study group, separately for original set 
of items as well as each of the various generated 
subsets of items, exploratory factor analysis was 
carried out. This involved principal component 
analysis as extraction method and orthogonal varimax 
(with Kaiser Normalization) as rotation method [14]. 
This exploratory analysis was used to guide the 
strategy regarding item reduction. Every time, 
Cronbach’s alpha as reliability coefficient for 
questionnaire was worked out. Further, Cronbach’s 
alpha in case an item is deleted was also examined 
(results not listed). Under this analysis, different 
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underlying factors (i.e., constructs or components) and 
items associated with them were also identified 
(Appendix 3). In case of more than one construct, one 
of the possible interpretations may be that CES 
questionnaire failed to completely measure quality of 
course, but did measure some related constructs [15]. 
In this case, for more appropriateness, the number of 
such constructs is expected to equate the number of 
sections/ visible broad features in the questionnaire. 
Also, logically and intuitively each of the identified 
constructs need to involve a different set of related 
items in order to consider them as sub — components of 
quality of course [15]. 


A critical comparative appraisal of results under 
exploratory analysis of various data sets, resulted 
from generated sub sets of items under second study 
group, could provide an optimized reduced set of 15 
items (i.e. reduced questionnaire). This questionnaire 
(Appendix 4) reflects relative desired merits in terms 
of reducing problem of collinearity among items; 
retaining relationship among them; involving genuine 
number of extracted factors (i.e., constructs or 
components), and also increased independence of 
extracted factors; and showing negligible change in 
overall reliability of questionnaire, as well as the 
proportion of total variance being explained by the 
underlying factors [3, 7, 8, 9, 14]. 


To add further strength to this exercise, as a measure to 
assess the consistency in pattern of observed results, 
this exploration was again repeated for remaining 
study groups (first, third & fourth). 


RESULTS 

As indicated by the observed determinant of matrix 
(Table 3), for each of the four study groups in college 
of nursing, there was genuine evidence regarding 
serious involvement of multicollinearity among the 
items of the original questionnaire. This further 
supports expressed views, by the college 
administrators as well as faculty, regarding the need of 
possible reduction of items in this questionnaire. For 
this, as mentioned earlier, only one of the two highly 
correlated (0.70 or higher correlation) items needs to 
be retained in the questionnaire. For each of the 
considered study groups in College of Nursing, the 
correlation matrix was explored (e.g. Appendix 2). The 
initial exploration involved second study group with 
largest pairs of highly correlated items (0.70 or higher 
correlation). Accordingly, as an added strength to 
exploratory analysis, this exploration relied on a large 
number of probable subsets of items. As a matter of 
fact, the subset of items without item 14 
comparatively involved more desired merits of CESQ 
than that with item 14 (results not listed). Finally, an 
optimized reduced set of 15 items could be identified 
that reflects relative desired merits in terms of reducing 
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problem of collinearity among items; retaining 7, 8, 9, 14] related to characteristics desired to assess 
relationship among them; involving increased adequacy of exploratory analysis and also the merits of 
appropriateness of extracted factors (i.e., constructs or a questionnaire. They are as: Kaiser-meyer-olkin 
components), and also increased independence of (KMO) measure of sampling adequacy, determinant of 
extracted factors; and showing negligible change in correlation matrix (status of collinearity), significance 
overall reliability of questionnaire, as well as the level for Bartlet’s test of sphericity (interrelationship of 
proportion of total variance being explained by the items), number of extracted components (eigen value 
underlying factors. In conclusion, the optimised >1) as indicative evidence towards unidimensionality 
reduced CES questionnaire is finalised to consist of of the questionnaire, % of total variance accounted by 
only 15 items (Appendix 4) instead of 24 items in the the extracted factors, and Cronbach’s alpha (overall 
original CES questionnaire (Appendix 1). reliability of questionnaire). Again, under each study 

group, these results foreach section (involving more 
For each study group, the results of factor analysis for than one item) of the questionnaires are compared 
original and reduced CES questionnaires are compared (Table 4). 


(Table 3). This table embodies the respective results 3, 


Table 3: Comparative results of factor analysis for original and reduced CES questionnaire 


Group | Questionnaire | 1* 2* 3* 4* | 5* 6* 

1 Original 0.969 | 1.44E-008 | 0.000 | 2 | 61.404 | 0.966 
Reduced 0.962 | 8.32E-005 | 0.000 | 1 | 56.886 | 0.946 

2 Original 0.979 | 2.46E-012 | 0.000 | 2 | 72.966 | 0.979 
Reduced 0.970 | 5.93E-007 | 0.000 | 1 | 67.787 | 0.965 

3 Original 0.976 | 3.28E-009 | 0.000 | 2. | 63.887 | 0.970 
Reduced 0.967 | 2.24E-005 | 0.000 | 1 | 60.587 | 0.953 

4 Original 0.959 | 1.77E-007 | 0.000 | 2 | 55.935 | 0.958 
Reduced 0.945 | 0.000 0.000 | 1 | 50.706 | 0.930 


* — 1: Kaiser-meyer-olkin measure of sampling adequacy 


2: Determinant of correlation matrix 

3: p-value, bartlett’s test of sphericity 

4: Number of factors extracted (eigen value >1) 
5: % of total variance explained by these factors 
6: Cronbach’s alpha 


Table 4: Groupwise Cronbach's Alpha by Section of the Questionnaire 


Group | Group 2 Group 3 Group 4 
# | Section OQ RQ OQ RQ OQ RQ OQ RQ 
1 | At the start of the course, I was made clear 0.334 | 0.743 | 0.896 | 0.825 | 0.860 | 0.770 | 0.821 | 0.710 
as : : : ' : ; : . 


2 | During this course, my instructors 


0.938 | 0.918 | 0.961 | 0.944 | 0.946 | 0.929 | 0.916 | 0.887 


= Ora AIP OUTS Ry CORE AEE, 0.802 | 0.619 | 0.866 | 0.720 | 0.818 | 0.737 | 0.775 | 0.616 


4 | Evaluation of the course: this course helped me 


to 0.869 0.902 0.893 0.866 


Combined 0.966 | 0.946 | 0.979 | 0.965 | 0.970 | 0.953 | 0.958 | 0.930 


OQ - Original Questionnaire, RQ - Reduced Questionnaire 
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Under analysis of original questionnaire in each study 
group, there were two extracted factors (Table 3, 
Appendix 3). It reflects neither unidimensionality of 
this questionnaire nor expected number of factors (five 
in the present case because of five sections in the 
questionnaire). But, consistently a higher magnitude of 
factor loading (£0.40) was observed for each item 
under each factor. However, these factors did not 
involve distinct set of items. Also, the related extracted 
communalities reflecting the amount of total variance 
in each item that can be explained by the retained 
factors [15] was consistently higher. On the other 
hand, under analysis of items in reduced questionnaire, 
there was only one extracted factor that reflects 
unidimensionality of the questionnaire. Also, 
comparatively a higher magnitude of factor loading 
(=0.60) was observed for each item. Interestingly, it 
remained true under each study group. Each of the four 
study groups revealed almost similar observations 
through analysis of original and _ reduced 
questionnaires. As an added observation, consistently a 
higher magnitude of factor loadings as well as 
communalities supports the validity of this exploratory 
analysis [3,7]. Further, Kaiser-meyer-olkin (KMO) 
measure of sampling adequacy (Table 3) also supports 
this view. 


The results across the considered study groups (Table 
3) reveal that the reduced questionnaire fulfils almost 
all the required attributes. For example, it retains the 
required interrelationship among the items. Further, in 
case of almost each study group, it reduces level of 
collinearity problem present in the original 
questionnaire. Also, in each study 
group, it provides only one extracted component as 
indicative evidence towards unidimensionality of the 
questionnaire (Table 3, Appendix 3). Further, for each 
study group, the original questionnaire results to two 
components that do not involve distinct set of items 
(Appendix 3). The proportion of total variance being 
explained by the underlying factors under reduced 
questionnaire remains at acceptable range (Table 3). 
Also, deletion of any individual item from the 
questionnaire does not make any change in reliability 
of the questionnaire (results not listed). Finally, the 
overall reliability of the reduced questionnaire almost 
remains almost same as that of the original 
questionnaire (Table 3). A comparison of section wise 
reliability also flows on same line (Table 4). It may be 
worthwhile to emphasize here that these explorations 
with existing CES data in college of nursing provided 
consistent results across the study groups. Hence, the 
number of items in reduced questionnaire remains as 
optimum. 


Thus, especially in an early phase of preparing for 
academic accreditation, the original CES questionnaire 
involving 24 items (Appendix 1) can be very well 
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replaced by this reduced CES questionnaire involving 
only 15 items (Appendix 4). 


SUMMARY AND CONCLUSIONS 

In summary, based on exploratory analysis of existing 
CES data, the reduction of items in the CES 
questionnaire from 24 to 15 does not distort its desired 
attributes. Incidentally it seems to involve better 
required scientific merits of a questionnaire. It retains 
the required interrelationship among the items; and 
also it reduces level of collinearity problem present in 
the original questionnaire. Also, interestingly it has 
indicative evidence towards unidimensionality of the 
questionnaire consistently. These observations suggest 
that reduced CES questionnaire is likely to measure 
quality of course at par with original questionnaire, if 
not more meaningfully. The proportion of total 
variance being explained by the underlying factors 
under reduced questionnaire also remains at acceptable 
range. 


As a matter of fact, deletion of any individual item 
from the original CES questionnaire did not make any 
noticeable change in overall reliability of the 
questionnaire (results not listed). Also, the overall 
reliability of the reduced questionnaire almost 
remained same as that of the original. It may be 
worthwhile to emphasize here that these explorations 
with existing CES data provided almost consistent 
results across four study groups. Hence, in our opinion, 
the number of items in reduced questionnaire remains 
as optimum. 


The excluded items from the 24-item original 
questionnaire incidentally do not capture all the items 
of any of the five areas of the questionnaire. In other 
words, each of the five aspects of the original 
questionnaire also remains intact in the reduced 
questionnaire. As a matter of fact, comparatively 
(Appendix 1 & 4), most of the excluded items also lack 
clarity at the level of students. Further, as required at 
this institution, reduced questionnaire will also reduce 
burden on the students. Hence, the use of optimized 
15- item questionnaire is likely to have better 
involvement of students that can provide more accurate 
and reliable data. It may thus be concluded that the 
optimized 15-item CES questionnaire (Appendix — 4) 
may be used without compromising much with 
structure and analytical power of the original CES 
questionnaire (Appendix 1). 


In the field of higher education, for the sake of its 
quality, consideration of the voices of students on 
issues related to enhancement and advancement of 
learning are equally important [16, 17, 18, 19, 20]. 
For this, one has to make all efforts to produce 
accurate results through a good evaluation process. 
Furthermore, an institution has to evolve a system of 
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regular evaluation of the evaluation system [21, 22, 23, 
24]. In this regard, we strongly believe that there is no 
better options than to listen every section of the 
university community (college administrators, faculty, 
students). As opined by them, possible reduction of 
items in existing course evaluation questionnaire is 
expected to be more appropriate for accurate and 
reliable evidence towards academic accreditation. In 
view of this, using exploratory analytical results of 
existing data, we could finalise optimised reduced CES 


questionnaire 


that is supported with exploratory 


scientific evidence. Like item reduction, institutions in 
higher education can very well make use of existing 
data in exploratory research to scientifically achieve 


other 


required objectives towards educational 


developments. 
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Appendix 1. Original CES questionnaire 
Items 
At the start of the course, I was made clear about: 
1. The course outline including the knowledge and skills 
2. The things I had to do to succeed, including assessment tasks and criteria for assessment 
3. Sources of help for me, including faculty office hours and reference material 
During this course, my instructors: 
4. Conducted the course consistent with the course outline 
5. Were available during office hours to help me 
6. Were enthusiastic about what they were teaching 
7. Cared about my academic progress, and, were helpful to me 
8. Were fully committed to the delivery. (E.g. On time start, regular presence, well prepared material etc) 
9. Had thorough knowledge of the content of the course 
10. Used up-to-date and useful course materials. (Texts, handouts, references etc.) 
11. Encouraged me to ask questions, and, develop my own ideas 
12. Inspired me to do my best work 
13. Made clear to me the links between this and other courses in my total program 
14. Gave fair grade to my Continuous Assessment [tests &assignments ] 
15. Gave the marks for the Continuous Assessment to me in time 
During this course, my college/department: 


16. Provided the resources I needed (textbooks, library, computers etc.) & made them available for me when I needed 
17. Provided effective technology to support my learning 

18. Made sure that the things to be done by me (class activities, assignments, laboratories etc), were appropriate for 
the knowledge & skills the course was intended to develop 

19. Required a fair amount of work for the credit hours allocated 

Evaluation of the course: this course helped me to: 

20. Learn what is important, &, will be useful to me 

21. Improve my ability to think & solve problems rather than memorize information 

22. Develop my ability to work as a member of a team 

23. Improve my ability to communicate effectively 

Overall evaluation 

24. Overall, I was satisfied with the quality of this course 
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Appendix 3: Factor loading on each item 


Group | Group 2 Group 3 Group 4 
Tent OQ RQ OQ RQ OQ RQ OQ RQ 
Paeieeel el — Pactae 1 rial Pasian ; actor all Factor 1 ail Factor 

1 0.72 0.72 0.81 0.89 0.68 0.41 0.78 0.44 0.66 0.75 
2 0.73 0.69 0.52 0.75 0.71 

3. 0.73 0.78 0.50 0.66 0.80 0.71 0.76 0.73 0.71 
4 0.68 0.41 0.79 0.64 0.56 0.85 0.74 0.78 0.67 0.72 
5 0.76 0.81 0.56 0.63 0.83 0.71 0.40 0.81 0.72 0.73 
6 0.77 0.76 860.71 0.81 0.75 0.76 0.62 0.70 
7 0.74 0.76 0.45 0.74 0.50 0.58 

8 0.67 0.74 0.87 0.60 0.69 0.42 0.79 0.73 0.66 
9 0.70 0.59 0.61 0.74 0.65 

10 0.60 0.46 0.76 0.74 0.44 0.86 0.68 0.44 0.81 0.59 0.47 0.76 
11 048 0.63 0.78 0.80 0.41 0.89 0.52 0.62 0.80 0.66 0.74 
12 0.56 0.54 0.78 0.73 0.47 0.87 0.59 0.54 0.80 0.66 0.72 
13. 0.45 0.63 0.74 0.80 0.87 0.48 0.65 0.79 0.68 0.72 
14. 0.56 0.49 0.74 0.81 0.83 0.50 0.54 0.74 0.60 0.66 
15 051 0.54 0.80 0.57 0.46 0.54 

16 0.56 0.44 0.73 0.50 0.58 0.75 0.46 0.57. 0.73 0.52 0.46 0.69 
17 0.68 0.73 0.53 0.53 0.74 0.47 0.54 0.72 0.58 0.66 
18 0.59 0.53 0.75 0.41 0.66 0.64 

19 0.54 0.50 0.80 0.49 0.51 0.67 

20 0.68 0.84 0.43 0.71 0.72 

21 0.78 0.72 0.70 0.43 0.82 0.76 0.78 0.72 0.70 
22 0.79 0.44 0.69 0.81 0.70 

23 0.81 0.44 0.75 0.81 0.68 

24 0.46 0.60 0.74 0.82 0.89 0.51 0.69 0.84 0.68 0.75 


Extraction Method: Principal Component Analysis 
Rotation Method: Varimax with Kaiser Normalization 
OQ - Original Questionnaire, RQ - Reduced Questionnaire 
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Appendix 4. Reduced CES questionnaire 

Items 

At the start of the course, I was made clear about: 

1. The course outline including the knowledge and skills 

2. Sources of help for me, including faculty office hours and reference material 
During this course, my instructors: 

3. Conducted the course consistent with the course outline 

4. Were available during office hours to help me 

5. Showed interest what they were teaching 

6. Were fully committed to the delivery. (E.g. On time start, regular presence, well prepared material, etc) 
7. Used up-to-date and useful course materials. (Texts, handouts, references etc.) 

8. Encouraged me to ask questions, and, develop my own ideas 

9. Inspired me to do my best work 

10. Made clear to me the links between this and other courses in my total program 
11. Gave the marks for the Continuous Assessment to me in time 

During this course, my college/department: 

12. Provided the resources I needed (textbooks, library, computers etc.) & made them available for me when I needed. 
13. Provided effective technology to support my learning 

Evaluation of the course: this course helped me to: 

14. Improve my ability to think & solve problems rather than memorize information 
Overall evaluation 

15. Overall, I was satisfied with the quality of this course 
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